Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Extensions to log server disk queue (made in the context of version vector/unicast) #11777

Open
wants to merge 2 commits into
base: version-vector-disk-queue
Choose a base branch
from
Open

Extensions to log server disk queue (made in the context of version vector/unicast) #11777

wants to merge 2 commits into from
+74 −6

Conversation

sbodagala
Copy link
Contributor

@sbodagala sbodagala commented Nov 13, 2024 •
edited
Loading

Goal:

The goal is to hold (version incompatible) disk queue related changes, made in the context of getting recovery to work with version vector, in a branch ("version-vector-disk-queue").

Changes:

Commit 37485d2:

  • Extend the disk queue entry data structure to hold the following information, needed by the recovery algorithm when unicast is enabled:
  • PrevVersion of a version
  • List of log servers that a commit proxy sends a commit version to

And, extend the code to populate "unknownCommittedVersions" list (the list of commit versions whose commit status is not known, and to be determined by the recovery version computation algorithm) on a log server restart.

Commit 0b7fbae:

Extend the log server restart logic to read versions from disk queue from known committed version onwards (instead of from the latest known persistent data durable version). The unicast recovery algorithm needs to know all commit versions, from known committed version onwards, that have been received by a log server in order to correctly compute the recovery version.

NOTE: Please note that these changes will cause version incompatibility and so additional code/logic will need to be added to make sure that upgrades/restart related simulation tests work properly.

Code-Reviewer Section

The general pull request guidelines can be found here.

Please check each of the following things and check all boxes before accepting a PR.

  • The PR has a description, explaining both the problem and the solution.
  • The description mentions which forms of testing were done and the testing seems reasonable.
  • Every function/class/actor that was touched is reasonably well documented.

For Release-Branches

If this PR is made against a release-branch, please also check the following:

  • This change/bugfix is a cherry-pick from the next younger branch (younger release-branch or main if this is the youngest branch)
  • There is a good reason why this PR needs to go into a release branch and this reason is documented (either in the description above or in a linked GitHub issue)

@sbodagala
- Extend the disk queue entry data structure to hold the following
37485d2 
information, needed by the recovery algorithm when unicast is enabled:
- PrevVersion of a version
- List of log servers that a commit proxy sends a commit version to

And, extend the code to populate "unknownCommittedVersions" list (the
list of commit versions whose commit status is not known, and to be
determined by the recovery version computation algorithm) on a log
server restart.

NOTE: Please note that these changes will cause version incompatibility
and so additional code/logic will need to be added to make sure that
upgrades/restart related simulation tests work properly.
@sbodagala
- On restart, make log servers read disk queue entries from known
0b7fbae 
committed version onwards in the case where the known committed version
is behind LogData::persistentDataDurableVersion (and version vector
unicast is enabled). This is so we will populate "LogData::unknownCommittedVersions",
on log server restart, with all versions that are needed by unicast recovery.
@foundationdb-ci
Copy link
Contributor

Result of foundationdb-pr-clang on Linux CentOS 7

  • Commit ID: 37485d2
  • Duration 0:35:14
  • Result: ❌ FAILED
  • Error: Error while executing command: ctest -j ${NPROC} --no-compress-output -T test --output-on-failure. Reason: exit status 8
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Contributor

Result of foundationdb-pr on Linux CentOS 7

  • Commit ID: 37485d2
  • Duration 0:44:16
  • Result: ❌ FAILED
  • Error: Error while executing command: ctest -j ${NPROC} --no-compress-output -T test --output-on-failure. Reason: exit status 8
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Contributor

Result of foundationdb-pr-clang-arm on Linux CentOS 7

  • Commit ID: 37485d2
  • Duration 0:46:40
  • Result: ❌ FAILED
  • Error: Error while executing command: ctest -j ${NPROC} --no-compress-output -T test --output-on-failure. Reason: exit status 8
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Contributor

Result of foundationdb-pr-macos-m1 on macOS Ventura 13.x

  • Commit ID: 37485d2
  • Duration 0:47:10
  • Result: ✅ SUCCEEDED
  • Error: N/A
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Contributor

Result of foundationdb-pr-macos on macOS Ventura 13.x

  • Commit ID: 37485d2
  • Duration 0:50:44
  • Result: ✅ SUCCEEDED
  • Error: N/A
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Contributor

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

  • Commit ID: 37485d2
  • Duration 0:56:11
  • Result: ✅ SUCCEEDED
  • Error: N/A
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)
  • Cluster Test Logs zip file of the test logs (available for 30 days)

@foundationdb-ci
Copy link
Contributor

Result of foundationdb-pr-clang on Linux CentOS 7

  • Commit ID: 0b7fbae
  • Duration 0:34:57
  • Result: ❌ FAILED
  • Error: Error while executing command: ctest -j ${NPROC} --no-compress-output -T test --output-on-failure. Reason: exit status 8
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Contributor

Result of foundationdb-pr on Linux CentOS 7

  • Commit ID: 0b7fbae
  • Duration 0:43:10
  • Result: ❌ FAILED
  • Error: Error while executing command: ctest -j ${NPROC} --no-compress-output -T test --output-on-failure. Reason: exit status 8
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Contributor

Result of foundationdb-pr-clang-arm on Linux CentOS 7

  • Commit ID: 0b7fbae
  • Duration 0:52:42
  • Result: ✅ SUCCEEDED
  • Error: N/A
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci
Copy link
Contributor

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

  • Commit ID: 0b7fbae
  • Duration 0:56:09
  • Result: ✅ SUCCEEDED
  • Error: N/A
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)
  • Cluster Test Logs zip file of the test logs (available for 30 days)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dlambrig dlambrig Awaiting requested review from dlambrig

@jzhou77 jzhou77 Awaiting requested review from jzhou77

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@sbodagala @foundationdb-ci